Lump Sum, Dollar Cost Averaging, or Buy the Dip?
An evaluation of three long term stock buying strategies.
This post is a Jupyter Notebook! Check out my GitHub to play with the source code.
Recently, I've seen a lot of chat online about the merits of lump sum investing in comparison with dollar cost averaging (DCA). For those of you who may not know, dollar cost averaging is the process of breaking a sum that you intend to invest into a few equal parts and then buying your asset of choiced at a fixed time interval. The basic idea behind doing this is that you'll average out the cost basis of your investment by avoiding a lump sum purchase either in at a peak or valley of the asset price. In short, DCA is used as a means of mitigating risk. There is another subset of people whose only advice is "buy the dip." I figured just for fun we could analyse that strategy alongside the other two.
To start, we can download a dataset to play with. I've gone ahead and pulled all the available daily price data for the S&P 500 ETF called SPY from Yahoo Finance. If you'd like you can check out the raw data here.
import random
import pandas as pd
import numpy as np
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from plotly.figure_factory import create_table
SPY_DATA_LOCATION = (
"https://raw.githubusercontent.com/borsboomT/dca_v_lump_v_dip/main/SPY.csv"
)
raw_df = pd.read_csv(SPY_DATA_LOCATION)
raw_df["Date"] = pd.to_datetime(raw_df["Date"])
create_table(raw_df.head())
The data has several columns, but what we're interested in is the adjusted close. This is the value of SPY at the close of each day, adjusted for splits and dividends. By using the adjusted close we get the best estimate of the underlying price movement of an asset.
Real World Analysis
To start, we'll do a simple experiment. What do we see if we invest $10000 using each of these strategies, using the start of our data as our entry point? In order to do this, we need to very clearly define how each of our strategies work:
- Lump Sum Investing
- All cash on hand is invested on the first available market day.
- DCA Investing
- Cash on hand is equally divided into a fixed number of investment portions.
- One of these portions is invested each week.
- Dip Investing
- There is very little in the way of actual systematic data for this, so I made some assumptions.
- We divide our total capital into the same number of portions as in the DCA strategy.
- We use one of those portions to buy the asset if today's adjusted close is equal to the thirty day rolling minimum.
We can then take a look at the number of SPY shares each strategy was able to accrue.
df = raw_df.copy(deep=True)
# Define our cash on hand, and the amount we're investing as a lump sum.
total_cash = 10000
lump_sum = total_cash
# Here we calculate the thirty day rolling minimum and determine the name of each market
# day for later use.
df["day_name"] = df["Date"].dt.day_name()
df["local_min"] = df["Adj Close"].rolling(window=30, min_periods=15).min()
# We will be breaking our investment into 10 portions for DCA averaging.
# 10 was used based on recommendations found online.
# The cash value of each DCA investment is the determined.
num_dca_installments = 10
dca_val = total_cash / num_dca_installments
# We make pandas series for each of the investment strategies.
# Lump investing has a single stock purchase on day 1, and no other purchases
df["lump_shares"] = pd.Series(dtype="float64")
df["lump_shares"].iloc[0] = lump_sum / df["Adj Close"].iloc[0]
df["lump_shares"].fillna(0, inplace=True)
# DCA has a purchase every monday, we'll exclude purchases beyond our total
# cash limit in a later step.
df["dca_shares"] = dca_val / df["Adj Close"][df["day_name"] == "Monday"]
# Dip has purchases whenever the adjusted close is equal to the local minimum.
df["dip_shares"] = (
dca_val / df["Adj Close"][df["Adj Close"] == df["local_min"]]
)
# We then sum up the number of shares purchased by each strategy, taking care
# to only include the proper number of purchases for the DCA and dip
# strategies.
total_lump_shares = df["lump_shares"].sum()
total_dca_shares = df["dca_shares"].dropna().head(num_dca_installments).sum()
total_dip_shares = df["dip_shares"].dropna().head(num_dca_installments).sum()
print("Lump Shares: {}".format(total_lump_shares))
print("DCA Shares: {}".format(total_dca_shares))
print("Dip Shares: {}".format(total_dip_shares))
It looks like lump sum investing gathered to most shares using the above analysis, but there's a glaring flaw in this analysis. We're using SPY data, and during the period of our analysis we had a bull market. Since the lump sum strategy bought its shares the earliest, it stands to reason it would have managed to buy the most shares.
In order to perform a proper analysis, we can add a couple techniques to our methodology to make it more robust. Those techniques include Monte Carlo analysis, as well as randomized entry and exit.
Monte Carlo Analysis
Monte Carlo Analysis a process by which a single data set can be resampled in order to generate a distribution of datasets. It can be a little tricky with time series data, but we'll go through how it can be done. We'll start by detrending the data using the method described in the book Evidence-Based Technical Analysis. These detrended time series are often used in order to evaluate algorithmic trading strategies outside the context of an overall market trend. We won't be using the detrended series in that way, because we're interested in the effect of the trend, but if we randomize the order of the log returns we can generate new time series data sets based on real price action information. We also need to remember to recalculate the local minimum for the simulated data, so that we can use it in our dip strategy.
If this part seems a little too complex, don't worry about it. Understanding how this works isn't important to the rest of the analysis, so long as you understand that it can be used to generate simulated time series data from real datasets.
def perform_monte_carlo(df):
df["log_ret"] = np.log(df["Adj Close"]) - np.log(df["Adj Close"].iloc[0])
df["log_ret_diff"] = df["log_ret"].diff()
df["detrended_close"] = df["log_ret_diff"] * df["Adj Close"].iloc[0]
df["log_ret_random"] = pd.Series(dtype="float64")
df["log_ret_random"][1:] = (
df["log_ret_diff"].iloc[1:].sample(frac=1).to_list()
)
df["detrended_monte_close"] = (
df["log_ret_random"] * df["Adj Close"].iloc[0]
)
df["log_ret_random_sum"] = df["log_ret_random"].cumsum()
df["simulated_close"] = (
np.exp(df["log_ret_random_sum"]) * df["Adj Close"].iloc[0]
)
df["simulated_close"].iloc[0] = df["Adj Close"].iloc[0]
df["simulated_local_min"] = (
df["simulated_close"].rolling(window=30, min_periods=15).min()
)
return df
df = perform_monte_carlo(df)
fig = make_subplots(rows=2, cols=1, shared_xaxes=True, vertical_spacing=0.02)
fig.add_trace(
go.Scatter(name="Raw Data", x=df["Date"], y=df["Adj Close"], mode="lines"),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="Monte Carlo Simulation",
x=df["Date"],
y=df["simulated_close"],
mode="lines",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="Detrended Data",
x=df["Date"],
y=df["detrended_close"],
mode="lines",
),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(
name="Detrended Monte Carlo",
x=df["Date"],
y=df["detrended_monte_close"],
mode="lines",
),
row=2,
col=1,
)
fig.update_xaxes(title_text="Date", row=2, col=1)
# Update yaxis properties
fig.update_yaxes(title_text="Adjusted SPY Value (USD)", row=1, col=1)
fig.update_yaxes(title_text="Detrended Spy Value", row=2, col=1)
fig.update_layout(
title="Raw and Simulated SPY Data",
showlegend=True,
)
fig.show()
Now this function can be used to generate tons of data for us to play with!
traces_range = range(1, 20)
fig = go.Figure()
fig.add_trace(go.Scatter(name="Original", x=df["Date"], y=df["Adj Close"]))
for i in traces_range:
df = perform_monte_carlo(df)
fig.add_trace(
go.Scatter(
name="Monte {}".format(i), x=df["Date"], y=df["simulated_close"]
)
)
fig.update_layout(
title="Bulk Simulated SPY Data",
showlegend=True,
xaxis_title="Date",
yaxis_title="Simulated Adjusted SPY Value (USD)",
)
fig.show()
Notice that each of these time series start and end at the same price. That's an intended function of how this process works, but it doesn't work for our analysis. In order to add an extra layer of randomness to our methodology we can introduce random entry and exit points.
Random Entry and Exit
By entering the market at a random time in our dataset, we better emulate how the average person approaches the market. Not everyone enters the marks on January 29th, 1993. In fact, you would be hard pressed to do that any time soon. Since the Monte Carlo process we introduced also ends at a fixed date with a fixed SPY value, we'll randomize the exit as well. We need to take special care here to ensure that the generated subset contains enough data to perform all the DCA purchases for our strategy.
def get_random_subset(df, num_dca_installments):
num_days = len(df)
random_entry_point = random.randint(0, num_days)
random_exit_point = random.randint(random_entry_point, num_days)
subset_df = df.copy(deep=True).iloc[
random_entry_point:random_exit_point, :
]
greater_than_one_week = (
subset_df["day_name"].str.lower().str.contains("monday").any()
)
if not greater_than_one_week:
subset_df = get_random_subset(df, num_dca_installments)
num_subset_weeks = subset_df["day_name"].value_counts()["Monday"]
if num_subset_weeks < num_dca_installments:
subset_df = get_random_subset(df, num_dca_installments)
return subset_df
trace_range = range(0, 10)
fig = go.Figure()
for i in trace_range:
subset_df = get_random_subset(df, num_dca_installments)
fig.add_trace(
go.Scatter(
name="Subset {}".format(i),
x=subset_df["Date"],
y=subset_df["Adj Close"],
)
)
fig.update_layout(
title="SPY Data Subsets",
showlegend=True,
xaxis_title="Date",
yaxis_title="Adjusted SPY Value (USD)",
)
fig.show()
It's a little hard to see because of how the data overlays here, but this function nicely returns random subsets of continuous data from the given time series. Now, we can combine the two techniques.
Combining Monte Carlo and Random Entry/Exit
This is a simple matter of making a function that nicely wraps up the other two functions.
def get_monte_carlo_subset(df, num_dca_installments):
df = perform_monte_carlo(df)
subset_df = get_random_subset(df, num_dca_installments)
return subset_df
trace_range = range(0, 10)
fig = go.Figure()
for i in trace_range:
subset_df = get_monte_carlo_subset(df, num_dca_installments)
fig.add_trace(
go.Scatter(
name="Simulated Subset {}".format(i),
x=subset_df["Date"],
y=subset_df["simulated_close"],
)
)
fig.update_layout(
title="Simulated SPY Data Subsets",
showlegend=True,
xaxis_title="Date",
yaxis_title="Simulated Adjusted SPY Value (USD)",
)
fig.show()
Perfect, it's so ugly that it's beautiful. Now we can use these generated datasets to create a distribution of test results for our three strategies! Let's start by wrapping our strategy evaluation code from earlier into a convenient function. We also need to add a little bit to the end in order to calculate the annualized returns for the trade, in order to make our different trial runs comparable with eachother.
def randomize_dataset_and_evaluate_strategies(
df, num_dca_installments, total_cash
):
subset_df = get_monte_carlo_subset(df, num_dca_installments)
dca_val = total_cash / num_dca_installments
# We make pandas series for each of the investment strategies.
# Lump investing has a single stock purchase on day 1, and no other purchases
subset_df["lump_shares"] = pd.Series(dtype="float64")
subset_df["lump_shares"].iloc[0] = (
lump_sum / subset_df["simulated_close"].iloc[0]
)
subset_df["lump_shares"].fillna(0, inplace=True)
# DCA has a purchase every monday, we'll exclude purchases beyond our total
# cash limit in a later step.
subset_df["dca_shares"] = (
dca_val
/ subset_df["simulated_close"][subset_df["day_name"] == "Monday"]
)
# Dip has purchases whenever the adjusted close is equal to the local minimum.
subset_df["dip_shares"] = (
dca_val
/ subset_df["simulated_close"][
subset_df["simulated_close"] == subset_df["simulated_local_min"]
]
)
# We then sum up the number of shares purchased by each strategy, taking care
# to only include the proper number of purchases for the DCA and dip
# strategies.
total_lump_shares = subset_df["lump_shares"].sum()
total_dca_shares = (
subset_df["dca_shares"].dropna().head(num_dca_installments).sum()
)
total_dip_shares = (
subset_df["dip_shares"].dropna().head(num_dca_installments).sum()
)
lump_cash = total_lump_shares * subset_df["simulated_close"].iloc[-1]
dca_cash = total_dca_shares * subset_df["simulated_close"].iloc[-1]
dip_cash = total_dip_shares * subset_df["simulated_close"].iloc[-1]
lump_returns = (lump_cash - total_cash) / total_cash
dca_returns = (dca_cash - total_cash) / total_cash
dip_returns = (dip_cash - total_cash) / total_cash
ann_exponent = 365 / len(subset_df)
annualized_lump = np.power(lump_returns + 1, ann_exponent) - 1
annualized_dca = np.power(dca_returns + 1, ann_exponent) - 1
annualized_dip = np.power(dip_returns + 1, ann_exponent) - 1
return [annualized_lump, annualized_dca, annualized_dip]
Now we can easily perform the analysis.
def get_test_result_df(
df, num_dca_installments, total_cash, num_random_entries
):
test_range = range(0, num_random_entries)
test_result_list = []
for i in test_range:
test_result = randomize_dataset_and_evaluate_strategies(
df, num_dca_installments, total_cash
)
test_result_list.append(test_result)
results_df = pd.DataFrame(test_result_list, columns=["lump", "dca", "dip"])
return results_df
num_random_entries = 10000
results_df = get_test_result_df(
df, num_dca_installments, total_cash, num_random_entries
)
create_table(results_df.head())
fig = go.Figure()
fig.add_trace(go.Histogram(name="Lump Sum", x=results_df["lump"]))
fig.add_trace(go.Histogram(name="DCA", x=results_df["dca"]))
fig.add_trace(go.Histogram(name="Dip", x=results_df["dip"]))
fig.update_traces(opacity=0.5)
fig.update_layout(
title="Simulation Results Histogram",
xaxis_title="Annualized Return (%)",
yaxis_title="Count",
showlegend=True,
barmode="overlay",
)
fig.show()
The histogram above shows a fairly normal distribution with very long tails. We can also see that the dip strategy has a significant collection of runs that resulted in complete liquidation of the portfolio.
fig = go.Figure()
fig.add_trace(
go.Box(
name="Lump Sum",
y=results_df["lump"],
)
)
fig.add_trace(
go.Box(
name="DCA",
y=results_df["dca"],
)
)
fig.add_trace(
go.Box(
name="Dip",
y=results_df["dip"],
)
)
fig.update_layout(
title="Simulation Results Box Plot",
yaxis_title="Annualized Return (%)",
showlegend=False,
barmode="overlay",
)
fig.show()
The box plot above showcases that the three strategies are quite similar for most trials, this is also seen quit well in the histogram. The box plot does a much better job of showcasing the propensity of outliers in the lump sum method to be quite favorable, and outliers for the dip method to be disastrous. Let's take a closer look at the median values for the three strategies.
fig = go.Figure()
fig.add_trace(
go.Bar(
x=["Lump Sum"],
y=[results_df["lump"].median()],
)
)
fig.add_trace(
go.Bar(
x=["DCA"],
y=[results_df["dca"].median()],
)
)
fig.add_trace(
go.Bar(
x=["Dip"],
y=[results_df["dip"].median()],
)
)
fig.update_layout(
title="Simulation Results Median Values",
yaxis_title=" Median Annualized Return (%)",
showlegend=False,
barmode="overlay",
)
fig.show()
Well, it looks like the dip strategy is just not feasible. As for the lump sum and DCA strategies, this data suggests that the lump sum investment strategy should be better! The median value of the lump sum strategy is slightly higher than the DCA value, the lump sum strategy has similar downside risk to the DCA strategy as seen in the box plot, and the lump sum strategy has larger upside prospects! Before we can conclusively say that the lump sum strategy is better, we need to investigate a couple more things.
Varying the Number of Installments
Maybe DCA performs better with a different number of installments! Lets take a look at what the data says. The looping gets a little odd here, but that's just me dealing with the idiosyncracies of Plotly in order to get the box plots to appear in a specific order.
Note: The lump sum strategy shouldn't be expected to change to much from trial to trial. The variation that we're seeing here is simply caused by variations in the monte carlo simulations.
dca_installment_test_range = range(5, 50, 5)
fig = go.Figure()
lump_trace_list = []
dca_trace_list = []
dip_trace_list = []
results_dict = dict()
for num_dca_installments in dca_installment_test_range:
results_dict[num_dca_installments] = get_test_result_df(
df, num_dca_installments, total_cash, num_random_entries
)
for num_dca_installments in results_dict:
results_df = results_dict[num_dca_installments]
lump_trace = go.Box(
name="Lump {}".format(num_dca_installments),
y=results_df["lump"],
legendgroup="Lump",
)
dca_trace = go.Box(
name="DCA {}".format(num_dca_installments),
y=results_df["dca"],
legendgroup="DCA",
)
dip_trace = go.Box(
name="Dip {}".format(num_dca_installments),
y=results_df["dip"],
legendgroup="Dip",
)
lump_trace_list.append(lump_trace)
dca_trace_list.append(dca_trace)
dip_trace_list.append(dip_trace)
for trace in lump_trace_list:
fig.add_trace(trace)
for trace in dca_trace_list:
fig.add_trace(trace)
for trace in dip_trace_list:
fig.add_trace(trace)
fig.update_layout(
title="DCA Installments Exploration Box Plot",
yaxis_title="Annualized Return (%)",
showlegend=False,
barmode="overlay",
)
fig.show()
This plot makes one thing very clear, the dip strategy should never actually be used. The comparison between the lump sum and dca strategies still needs clarification. What we really need to look at is the median return, the downside risk, and the upside potential for each of the trials.
risk_comparison_list = []
for num_dca_installments in results_dict:
results_df = results_dict[num_dca_installments]
lump_max = results_df["lump"].max()
lump_min = results_df["lump"].min()
lump_median = results_df["lump"].median()
dca_max = results_df["dca"].max()
dca_min = results_df["dca"].min()
dca_median = results_df["dca"].median()
risk_comparison_list.append(
[
num_dca_installments,
lump_max,
lump_min,
lump_median,
dca_max,
dca_min,
dca_median,
]
)
risk_comparison_df = pd.DataFrame(
risk_comparison_list,
columns=[
"num_dca_installments",
"lump_max",
"lump_min",
"lump_median",
"dca_max",
"dca_min",
"dca_median",
],
)
fig = make_subplots(rows=3, cols=1, shared_xaxes=True, vertical_spacing=0.02)
fig.add_trace(
go.Scatter(
name="Lump Max",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_max"],
mode="lines",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="Lump Median",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_median"],
mode="lines",
),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(
name="Lump Min",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_min"],
mode="lines",
),
row=3,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Max",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_max"],
mode="lines",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Median",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_median"],
mode="lines",
),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Min",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_min"],
mode="lines",
),
row=3,
col=1,
)
fig.update_xaxes(title_text="Number of DCA Installments", row=3, col=1)
# Update yaxis properties
fig.update_yaxes(title_text="Annualized Return (%)", row=2, col=1)
fig.update_layout(
title="Lump Sum vs DCA Risk Assessment",
showlegend=True,
)
fig.show()
This is a pretty useful comparison. The lump sum values are favorable to the DCA values for the median and maximum, whereas the DCA values are favorable for the minimum. I think we need to ask one more question before offering a final conclusion, how do these strategies perform in aggressive bear markets?
Bear Market Analysis
Let's redo that final analytical step using data from an inverse SPY ETF as a confirmation of what we suspect. We'll use the inverse SPY ETF SH for this.
SH_DATA_LOCATION = (
"https://raw.githubusercontent.com/borsboomT/dca_v_lump_v_dip/main/SH.csv"
)
raw_df = pd.read_csv(SH_DATA_LOCATION)
raw_df["Date"] = pd.to_datetime(raw_df["Date"])
create_table(raw_df.head())
df = raw_df.copy(deep=True)
df["day_name"] = df["Date"].dt.day_name()
dca_installment_test_range = range(5, 50, 5)
results_dict = dict()
for num_dca_installments in dca_installment_test_range:
results_dict[num_dca_installments] = get_test_result_df(
df, num_dca_installments, total_cash, num_random_entries
)
risk_comparison_list = []
for num_dca_installments in results_dict:
results_df = results_dict[num_dca_installments]
lump_max = results_df["lump"].max()
lump_min = results_df["lump"].min()
lump_median = results_df["lump"].median()
dca_max = results_df["dca"].max()
dca_min = results_df["dca"].min()
dca_median = results_df["dca"].median()
risk_comparison_list.append(
[
num_dca_installments,
lump_max,
lump_min,
lump_median,
dca_max,
dca_min,
dca_median,
]
)
risk_comparison_df = pd.DataFrame(
risk_comparison_list,
columns=[
"num_dca_installments",
"lump_max",
"lump_min",
"lump_median",
"dca_max",
"dca_min",
"dca_median",
],
)
fig = make_subplots(rows=3, cols=1, shared_xaxes=True, vertical_spacing=0.02)
fig.add_trace(
go.Scatter(
name="Lump Max",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_max"],
mode="lines",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="Lump Median",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_median"],
mode="lines",
),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(
name="Lump Min",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["lump_min"],
mode="lines",
),
row=3,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Max",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_max"],
mode="lines",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Median",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_median"],
mode="lines",
),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(
name="DCA Min",
x=risk_comparison_df["num_dca_installments"],
y=risk_comparison_df["dca_min"],
mode="lines",
),
row=3,
col=1,
)
fig.update_xaxes(title_text="Number of DCA Installments", row=3, col=1)
# Update yaxis properties
fig.update_yaxes(title_text="Annualized Return (%)", row=2, col=1)
fig.update_layout(
title="Lump Sum vs DCA Risk Assessment - Bear Market",
showlegend=True,
)
fig.show()
Well there you have it. Under incredibly poor market conditions lump sum investments manage to keep their upside, but DCA is far better at preserving capital. The median and minimum values for the DCA are much more favorable under these conditions. It appears that there is also some advantage to be gained here by increasing the number of DCA installments.
The Last Word
DCA does appear to mitigate risk in bear markets, especially as the number of DCA installments is increased. There appears to be a somewhat diminishing return to breaking your investments into more than 20 installments in terms of added security. If you can stomach waiting 20 weeks to get all of your money into the market, this appears to be the way to go. When it comes to the median values for the returns there really isn't too big a difference between the strategies, you might pick up an extra percent annualized by using lump sum investment in a bull market. The benefits that DCA offers in a bear market should not be overlooked though, especially considering the current state of the globe.
I hope that was helpful to some of you folks. As always, I recommend that everyone who feels inclined download the data and have some fun with it yourself!